Variational Learning for Multi-Layer Networks of Linear Threshold Units
نویسنده
چکیده
Linear threshold units (LTUs) were originally proposed as models of biological neurons. They were widely studied in the context of the perceptron (Rosenblatt, 1962). Due to the difficulties of finding a general algorithm for networks with hidden nodes, they never passed into general use. In this work we derive an algorithm in the context of probabilistic models and show how it may be applied in multi-layer networks of LTUs. We demonstrate the performance of the algorithm on three data-sets.
منابع مشابه
Growing Layers of Perceptrons : Introducing the Extentron Algorithm
The ideas presented here are based on two observations of perceptrons: (1) when the perceptron learning algorithm cycles among hyperplanes, the hyperplanes may be compared to select one that gives a best split of the examples, and (2) it is always possible for the perceptron to build a hyper-plane that separates at least one example from all the rest. We describe the Extentron which grows multi...
متن کاملConvex Neural Networks
Convexity has recently received a lot of attention in the machine learning community, and the lack of convexity has been seen as a major disadvantage of many learning algorithms, such as multi-layer artificial neural networks. We show that training multi-layer neural networks in which the number of hidden units is learned can be viewed as a convex optimization problem. This problem involves an ...
متن کاملStudies with a Generalized Neuron Based PSS on a Multi-Machine Power System
An artificial neural network can be used as an intelligent controller to control non-linear, dynamic system through learning. It can easily accommodate non-linearities and time dependencies. Most common multi-layer feed-forward neural networks have the drawbacks of large number of neurons and hidden layers required to deal with complex problems and require large training time. To overcome these...
متن کاملA New Modification of the Reconstruction of Variational Iteration Method for Solving Multi-order Fractional Differential Equations
Fractional calculus has been used to model the physical and engineering processes that have found to be best described by fractional differential equations. For that reason, we need a reliable and efficient technique for the solution of fractional differential equations. The aim of this paper is to present an analytical approximation solution for linear and nonlinear multi-order fractional diff...
متن کاملHardness Results for General Two-Layer Neural Networks
We deal with the problem of learning a general class of 2-layer neural networks in polynomial time. The considered neural networks consist of k linear threshold units on the hidden layer and an arbitrary binary output unit. We show NP-completeness of the consistency problem for classes that use an arbitrary set of binary output units containing a function which depends on all input dimensions. ...
متن کامل